Understanding covariate shift in model performance
نویسندگان
چکیده
Three (3) different methods (logistic regression, covariate shift and k-NN) were applied to five (5) internal datasets and one (1) external, publically available dataset where covariate shift existed. In all cases, k-NN's performance was inferior to either logistic regression or covariate shift. Surprisingly, there was no obvious advantage for using covariate shift to reweight the training data in the examined datasets.
منابع مشابه
Understanding covariate shift in model performance [ version
Three (3) different methods (logistic regression, covariate shift and k-NN) were applied to five (5) internal datasets and one (1) external, publically available dataset where covariate shift existed. In all cases, k-NN’s performance was inferior to either logistic regression or covariate shift. Surprisingly, there was no obvious advantage for using covariate shift to reweight the training data...
متن کاملSimultaneous Monitoring of Multivariate Process Mean and Variability in the Presence of Measurement Error with Linearly Increasing Variance under Additive Covariate Model (RESEARCH NOTE)
In recent years, some researches have been done on simultaneous monitoring of multivariate process mean vector and covariance matrix. However, the effect of measurement error, which exists in many practical applications, on the performance of these control charts is not well studied. In this paper, the effect of measurement error with linearly increasing variance on the performance of ELR contr...
متن کاملRobust Covariate Shift Prediction with General Losses and Feature Views
Covariate shift relaxes the widely-employed independent and identically distributed (IID) assumption by allowing different training and testing input distributions. Unfortunately, common methods for addressing covariate shift by trying to remove the bias between training and testing distributions using importance weighting often provide poor performance guarantees in theory and unreliable predi...
متن کاملAdaptive learning with covariate shift-detection for motor imagery-based brain-computer interface
A common assumption in traditional supervised learning is the similar probability distributionof data between the training phase and the testing/operating phase. When transitioning from the training to testing phase, a shift in the probability distribution of input data is known as a covariate shift. Covariate shifts commonly arise in a wide range of real-world systems such as electroencephalog...
متن کاملAnalysis of Kernel Mean Matching under Covariate Shift
In real supervised learning scenarios, it is not uncommon that the training and test sample follow different probability distributions, thus rendering the necessity to correct the sampling bias. Focusing on a particular covariate shift problem, we derive high probability confidence bounds for the kernel mean matching (KMM) estimator, whose convergence rate turns out to depend on some regularity...
متن کامل